[no squash] Auto-toggle TouchScreenGUI in-game when receiving touch/mouse input #14542

grorp · 2024-04-13T17:02:48Z

With this PR, the touch controls are automatically created/deleted based on the last used pointer type (mouse or touch).

It is based on #14472 because before that PR, TouchScreenGUI didn't ever delete its IGUIButtons, it just hid them. They were deleted somewhere else on shutdown. While this worked without in-game creation/deletion of TouchScreenGUIs, it would result in memory leaks with this PR.

Note that no automatic switching is done on keyboard input. This is because TouchScreenGUI + keyboard works while TouchScreenGUI + mouse doesn't.

Related: #14400, #14400 (comment), #14400 (comment)

To do

This PR is a Ready for Review.

Should automatic switching change the enable_touch setting as well?
Touchscreen broken on X11 due to fake mouse events
Camera jumps when switching from touchscreen to mouse on Android
Code cleanup
More testing on different platforms

How to test

Buy an Android phone, a bluetooth mouse and a bluetooth keyboard. See that the touch controls are automatically created/deleted when you use the touchscreen/mouse.

grorp · 2024-05-05T17:55:43Z

I've finally been able to fix the camera jump issue on Android using two changes from MoNTE48/irrlicht and one own change. Idk why this combination of changes works, but it does work and I've spent too many hours on this bug already.

The remaining hard issue is the "fake mouse events on X11" one.

grorp · 2024-05-05T17:56:55Z

irr/src/CIrrDeviceSDL.cpp

+				MouseY = irrevent.MouseInput.Y = SDL_event.motion.y;
+			} else {
+				MouseX = irrevent.MouseInput.X = MouseX + SDL_event.motion.xrel;
+				MouseY = irrevent.MouseInput.Y = MouseY + SDL_event.motion.yrel;


does this make a difference on platforms other than Android? at least Multicraft Irrlicht also applies it on all platforms: https://github.com/MoNTE48/Irrlicht/blob/a2910941a0c64e58f294e3e4aa041fb6f7a1db9a/source/Irrlicht/CIrrDeviceSDL.cpp#L856-L865

grorp · 2024-05-15T20:26:30Z

This PR now implements the approach suggested by @okias since my own idea didn't work as well:

Phase 1

Split touchscreen settings into UI layout and touchscreen controls

Phase 2

auto-detect form-factor and set newly created UI layout variable by default regarding to the detected device form-factor

Phase 3

switch touchscreen controls option from ON/OFF to AUTO/ON/OFF, where default auto toggles touchscreen controls on touch/mouse input

slightly modified, original by okias in #14400 (comment)

I've also renamed TouchScreenGUI to TouchControls to avoid confusion. rubenwardy has made a similar suggestion in #14075 (comment).

@okias Would you be interested in giving my changes a review? There's still some cleanup of accumulated quick-and-dirty fixes to do, but the PR should be ready in terms of concept and features now.

src/defaultsettings.cpp

okias · 2024-05-19T04:53:52Z

s/gui_touch/adjust_touchscreen_ui/ or /touchscreen_ui_adjust/... gui_touch feel too confusing.

But I hope after next release there will be rewrite, where UI will become hack-free for touchscreen and just universal code for both desktop and mobile.

to avoid confusion between touchscreen-related settings that affect GUIs (formspecs) and touchscreen-related settings that affect the touch controls (TouchControls / formerly TouchScreenGUI)

…ouch_gui touch_gui provide adjustment to the interface, so it's more touch friendly Signed-off-by: David Heidelberg <david@ixit.cz>

…ompiles again, but not tested yet)

grorp · 2024-06-13T17:36:44Z

A part of this has been split off into #14749. I've rebased this PR on top of that one.

I also did the remaining code cleanup / integration, so this is ready for review now. I quickly tested it on Android and it still seems to work.

irr/include/IEventReceiver.h

sfan5 · 2024-06-13T19:02:19Z

irr/include/IEventReceiver.h

@@ -344,6 +344,9 @@ struct SEvent

 		//! Type of mouse event
 		EMOUSE_INPUT_EVENT Event;
+
+		//! Is this a simulated mouse event generated by Minetest itself?
+		bool Simulated;


you could reuse e.g. ButtonStates with a flag that indicates simulated or not
but other than that setting this in the constructor sounds like the correct approach

you could reuse e.g. ButtonStates with a flag that indicates simulated or not

I fear bugs, consider this:

minetest/src/gui/guiFormSpecMenu.cpp

Lines 4122 to 4126 in 9def45a

// Mouse wheel and move events: send to hovered element instead of focused

if (event.EventType == EET_MOUSE_INPUT_EVENT &&

(event.MouseInput.Event == EMIE_MOUSE_WHEEL ||

(event.MouseInput.Event == EMIE_MOUSE_MOVED &&

event.MouseInput.ButtonStates == 0))) {

but other than that setting this in the constructor sounds like the correct approach

I now notice that setting it in the constructor probably isn't good either. Consider CIrrDeviceSDL, which reuses one SEvent variable for all Irrlicht events it creates from SDL events.

minetest/irr/src/CIrrDeviceSDL.cpp

Lines 646 to 650 in bc23a61

bool CIrrDeviceSDL::run()

{

os::Timer::tick();

SEvent irrevent;

Since union members other than MouseInput are used on that variable (at least KeyInput & TouchInput), MouseInput.Simulated might (in theory) already be overwritten when a mouse event is created.

I now notice that setting it in the constructor probably isn't good either. Consider CIrrDeviceSDL, which reuses one SEvent variable for all Irrlicht events it creates from SDL events.

bad practice IMO, it shouldn't be doing that.
if you want to fix that you could simply insert irrevent = {} before every use

Hmm, shouldn't replacing every occurrence of SEvent event; with SEvent event{}; also work as a solution?

grorp added Android Feature ✨ PRs that add or enhance a feature @ Client / Controls / Input Waiting (on dependency) Waiting on another PR or external circumstances (not for rebases/changes requested) labels Apr 13, 2024

grorp marked this pull request as draft April 13, 2024 17:04

grorp force-pushed the auto-switch branch 2 times, most recently from e3adfb7 to f4ca1ea Compare April 17, 2024 14:04

grorp removed the Waiting (on dependency) Waiting on another PR or external circumstances (not for rebases/changes requested) label Apr 17, 2024

okias mentioned this pull request Apr 21, 2024

Basic tablet/phone/watch autodetection #14400

Merged

grorp force-pushed the auto-switch branch from f4ca1ea to 956d05e Compare April 30, 2024 11:53

grorp linked an issue May 5, 2024 that may be closed by this pull request

Allow switching between touchscreen and other input on Desktop and Android #10062

Open

grorp force-pushed the auto-switch branch from 956d05e to e877d64 Compare May 5, 2024 17:46

grorp commented May 5, 2024

View reviewed changes

grorp force-pushed the auto-switch branch from 967b759 to 57cf513 Compare May 15, 2024 09:49

grorp mentioned this pull request May 15, 2024

Falling blocks stay immortals #14666

Closed

grorp force-pushed the auto-switch branch 4 times, most recently from 9d37510 to da9f2ea Compare May 15, 2024 19:55

grorp changed the title ~~Auto-toggle TouchScreenGUI in-game when receiving touch/mouse input~~ [no squash] Auto-toggle TouchScreenGUI in-game when receiving touch/mouse input May 15, 2024

grorp commented May 15, 2024

View reviewed changes

src/defaultsettings.cpp Outdated Show resolved Hide resolved

grorp force-pushed the auto-switch branch from cabb5af to b5c7d94 Compare May 17, 2024 11:10

grorp mentioned this pull request May 17, 2024

Update SDL for mingw builds #14678

Merged

grorp force-pushed the auto-switch branch from b5c7d94 to c077380 Compare May 21, 2024 20:28

okias mentioned this pull request Jun 12, 2024

SDL2: Support highdpi #14703

Merged

grorp mentioned this pull request Jun 12, 2024

[no squash] Split touch controls from UI #14749

Open

Rename TouchScreenGUI -> TouchControls

266137a

to avoid confusion between touchscreen-related settings that affect GUIs (formspecs) and touchscreen-related settings that affect the touch controls (TouchControls / formerly TouchScreenGUI)

grorp added 4 commits June 12, 2024 14:50

split enable_touch to touch_controls (for touchscreen controls) and t…

e467fa5

…ouch_gui touch_gui provide adjustment to the interface, so it's more touch friendly Signed-off-by: David Heidelberg <david@ixit.cz>

Auto-toggle TouchControls in-game when receiving touch/mouse input (c…

71c05c6

…ompiles again, but not tested yet)

Fix Android camera jump issue (phew)

348d003

Avoid bug on Linux/X11

c6978db

grorp force-pushed the auto-switch branch from c077380 to c6978db Compare June 13, 2024 13:39

grorp marked this pull request as ready for review June 13, 2024 17:34

grorp added the Waiting (on dependency) Waiting on another PR or external circumstances (not for rebases/changes requested) label Jun 13, 2024

grorp commented Jun 13, 2024

View reviewed changes

irr/include/IEventReceiver.h Outdated Show resolved Hide resolved

Manually set Simulated = false where necessary (fixup)

f74fff3

sfan5 reviewed Jun 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[no squash] Auto-toggle TouchScreenGUI in-game when receiving touch/mouse input #14542

[no squash] Auto-toggle TouchScreenGUI in-game when receiving touch/mouse input #14542

grorp commented Apr 13, 2024 •

edited

Loading

grorp commented May 5, 2024

grorp May 5, 2024

grorp commented May 15, 2024

okias commented May 19, 2024 •

edited

Loading

grorp commented Jun 13, 2024

sfan5 Jun 13, 2024

grorp Jun 14, 2024

sfan5 Jun 15, 2024 •

edited

Loading

grorp Jun 15, 2024

	// Mouse wheel and move events: send to hovered element instead of focused
	if (event.EventType == EET_MOUSE_INPUT_EVENT &&
	(event.MouseInput.Event == EMIE_MOUSE_WHEEL \|\|
	(event.MouseInput.Event == EMIE_MOUSE_MOVED &&
	event.MouseInput.ButtonStates == 0))) {

	bool CIrrDeviceSDL::run()
	{
	os::Timer::tick();

	SEvent irrevent;

[no squash] Auto-toggle TouchScreenGUI in-game when receiving touch/mouse input #14542

Are you sure you want to change the base?

[no squash] Auto-toggle TouchScreenGUI in-game when receiving touch/mouse input #14542

Conversation

grorp commented Apr 13, 2024 • edited Loading

To do

How to test

grorp commented May 5, 2024

grorp May 5, 2024

Choose a reason for hiding this comment

grorp commented May 15, 2024

okias commented May 19, 2024 • edited Loading

grorp commented Jun 13, 2024

sfan5 Jun 13, 2024

Choose a reason for hiding this comment

grorp Jun 14, 2024

Choose a reason for hiding this comment

sfan5 Jun 15, 2024 • edited Loading

Choose a reason for hiding this comment

grorp Jun 15, 2024

Choose a reason for hiding this comment

grorp commented Apr 13, 2024 •

edited

Loading

okias commented May 19, 2024 •

edited

Loading

sfan5 Jun 15, 2024 •

edited

Loading